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ABSTRACT 

A recursive procedure is derived for decoding of rate R— l/n binary con- 
volutional codes which minimizes the probability of the individual decoding 
(3,ecisions for each information bit subject to the constraint that the decoding 
delay be limited to A branches. This new decoding algorithm is similar to, but 
somewhat more complex than, the Viterbi decoding algorithm. A real-time, i.e. 
fixed decoding delay, version of the Viterbi algorithm is also developed and used 
for comparison to the new algorithm on simulated channels. It is shown that the 
new algorithm offers advantages over Viterbi decoding in soft— decision applications 
such as in the inner coding system for concatenated coding. 
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I. Introduction 

A binary convolutional code of rate R=l/n and memory order m can be encoded 
using a shift-register of m stages together vith n modulo-two adders as shown in 
Figxire 1. In this diagram a^ is the 


0 

12 m 




Fig. 1 An Encoder for a Rate R=l/n Convolutional Code of 
Memory Order m 


input information digit at time t and 

bj. = 

is the encoded branch at time t consisting of the n digits formed by the modulo- 
two adders in the encoder. We assume that the encoded digits are transmitted 
serially over a discrete memoryless channel (or DMC) and we let 

^ ~ ^^tl’’^t2’’ * ’^tn^ 
denote the corresponding received branch . 

The encoding shift-register is initially loaded with zeroes after which 

Ui^a^i—.a^ are encoded and then followed by ra zeroes (i.e. 

to clear the encoder . L is called the frame length . The case L=«> is not without 
interest and in fact most practical threshold decoders for convolutional codes 
have operated with an infinite frame length or, as it is usually stated, without 
resynchroni zation of the encoder. Sequential decoders on the other hand have 
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always "been employed with rather small frame lengths (on the order of 10m.) 

We shall let ^={a^,a^^^,. . .a^, ] denote the information sequence over 

time units t through t' inclusive, and similarly for and r^_^_ The 

Viterhi decoding algorithm [l], which is the maximum likelihood decoding algorithm 
for a convolutionally— coded frame sent over a discrete memoryless channel chooses 

A. 

as its estimate a^^ of the information sequence that sequence which maximizes 
the conditional probability 

and hence this algorithm minimizes the frame-error probability 

which is. the probability of the event that at least one information bit in the 
frame is wrongly decoded. 

For any interesting channel, it must be true that 


^FE " ^ 


so that Ppg is not a meaningful optimality criterion for large frames. The 
criterion of real interest in all cases is the bit-error probability 

^BE ^ L ^ 
t=l 

which gives the fraction of information digits which are wrongly decoded. From 
(l) and (2) it follows that 


^BE - ^FE - ^ ^BE 

so that when L is fairly small it makes little difference whether P or P is 

BE FE 

minimized (which explains the appropriateness of Viterbi decoding when L is small.) 
The bit-error probability, P^^, is minimized by the decoding rule which for 

A 

each t, lcb<L, chooses its estimate a^ as the binary digit which maximizes the 
conditional probability 

^^\l-[l,L+m]^* 
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Algorithms, similar to Viterhi^s, to accomplish this maximization have been pro- 
posed independently by Bahl-Cocke-Jelinek-Raviv 1 2] and by McAdam-Welch-Veber l3]. 
These algorithms require receipt of tlte entire frame r^ before decoding 

begins and so cannot be used vithout resynchronization. Moreover their implementa- 
tion requires storage which grows linearly in L and hence are practical alterna- 
tives to Viterbi decoding only for small L. 

In the following section we derive a recursive procedure for "real-time 
minimum— bit— error probability decoding" of rate R=l/n convolutional codes to 

minimize P under the constraint that the decoding delay be limited to A branches. 

BE 

In section III we formulate the corresponding decoding algorithm and show that its 
storage requirements are independent of L. For comparison purposes, we formulate 
a "real-time" modified Viterbi decoding algorithm in Section IV. Section V. reports 
the results of using these decoding procedures on a simulated additive white 
Gaussian noise channel. It is concluded that the improvement in for the real- 
time minimum— bit— error probability decoding algorithm is not enough to Justify 
the added complexity compared to Viterbi decoding but , as shown in Section VI , the 
new algorithm offers advantages in soft-decision applications such as concatenated 
coding , 


II. Derivation of a Real-Time Minimum-Bit -Error Probability Decoding Procedure, 

As in all previous optimal (in some sense) decoding procedures for convolu- 
tional codes, we shall make important use of the encoder state which at time t is 
defined as the contents of the shift-register in Fig. 1 i.e. the m-tuple of 
past information bits 

and where, by our convention, a^— 0 for x<0 and i>L« As the term state implies, 

s completely accounts for the past history of the encoder input in the sense that 
"t 



h 


and the input segment 5'j^-j. uniquely determine the output segment 
By conditioning on the encoder state, the calculation of , the probabilities re- 
quired for the decision rule can be simplified 

The decoding rule which minimizes under the "real-time” constraint that 

A 

a^ be decided from r^j-^ is that which chooses 


= 0 if 

P(a^=0|rfi^^^^]) 



(U) 


and chooses a^=l otherwise (where we have arbitrarily resolved ties in favor of 

/S 

the decision a =0.) Since 

u 


P(a =0|r _ 


and since the probabilities on the righthand side of this latter equation can be 
expressed as the summation over all states of the joint probabilities including 
the state, we have 


P(a =0|j 


I P(a,=( 


( 5 ) 


We now proceed to develop recursive formulas for the two probabilities 
appearing on the righthand side of (5)« 

For any t, t > 1, we may write 


But also 

where we have made use of the state property and our assumption that the channel 


is memoryless. Writing 



5 




t+1 ' t 


■f t’ t+1 


( 8 ) 


we then use the fact seen from (3) that the state = I\ ’^-1* * * 

has only two possible predecessors namely f^t-1 ’ ' ‘ * ’^t-m+1 ^ ^t-1 ’ * * * *^t-m+l ’ 


to write 


Pf I , = i 

^t+l*^t 0 otherwise 


( 9 ) 


where here and hereafter we write P(s) for the set containing the two possible 

predecessors of a state s. Finally, we note that,, for s_j_eP(s^_j_^) , since 

specifies a. (by its first digit) then it follows that the encoded branch ^ is 

imiquely determined by s and s [and we write b(s ,s ) for this branch] so that 

*t+J- Xf “ "C x+x 

and we note that this quantity is determined by the channel transition probabilities 
Substituting (7), (8), (9) and (lO) into (6) we have our desired recursion 


The starting value PCrj'j^ l]’®2^ ~ P(r.,S 2 ) needed to apply the recursion is simply 

Cl2) 


i P(rJb(0,s„)) if OeP(s ) 
Pt^.Sg) = {3 ^ ^ 


otherwise 

where we have used the fact that s^=0^. 

By an entirely analogous argument whose details we omit, the following 
recursion for the other probability on the righthand side of (5) may be obtained: 


^^^t’-[l,t+i]’^t+i+l^ ^ 2 ^^^+il-^^t+i’^t+i+l^^^^^t’^l,t+i-l]’®t+i^ 

which we shall use for 1 < i < A. The starting value needed for this recursion is 


P(a^,^l^t]’^t+1^ ^^^l,t] "®t+l^^^^t l^l,t] ’^t+1^ 

P(i'r-. u.i»Su. -,) if a_^ is the first digit of s . _ 
_ , "iljtJ t+1 t t+1 

^ X r\ — j,*u -w 


otherwise 


ilk) 
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so that the quantities obtained from the recursion (ll) directly provide the 
necessary initial conditions (l4) to he used with the recursion (l3). Hence ve 
have nov obtained a complete recursive procedure for performing real-time 
minimum-bit- err or probability decoding of convolutional codes . 


Ill, Implementation of the Decoding Algorithm 

We now describe an algorithm for implementing the decoding procedure whose 
recursive basis was developed in the previous section. Our algorithm requires the 
storage of two real numbers for each of the 2^ encoder states. We denote these 
stored quantities for state s as f(s) and g(s). At time t (in the algorithm) the 
first of these quantities will store the value 

f(s) = P(r[i^^pSt^^=s) 0 . 6 ) 

and the second, will store the value 

,g(s) = ClT ) 

We shall store the previous A values of this latter quantity summed over all 
states and denote these stored values as G^ 5 G 2 ,...G^ so that 

for l£i^A will be the stored values at time t (in the algorithm). It follows 

then that G. and 
A 


Z f(s) = E 


(19) 


are the desired numerator and denominator on the- righthand side of (5) for the 
decoding of a, . which is accomplished at time t (in the algorithm.) The only 
additional storage required is that for the received branches »r^^^ s • ■ • 

The recursions (ll) and (l^) directly correspond to the following: 

The Real-Time Minimum-Bit-Error Probability Decoding Algorithm 
Step 0 : Set t^l, set f ( s )— P(i^|, ]^(0_,s )) for the two states s having as a pre- 

decessor, and set f(s)=0 for all other states. Set G^=0 for l^i^ A. 
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Step l i Set g(s)=f(s) if the first digit of s is a 0 and set g(s)==0 otherwise. 

Step 2: For i=l,2,...A» make the replacement 

g(s) I ^ P(r, . Jh(s\s))g(sM for all s. 
s*eP(s) ^ 

A 

Step 3 : If t 5 A, go to step 2 . Otherwise set 

I f(s)/G^ >1 
s 

and set a, .=1 otherwise. 

X“ A 

step h : Increase t hy 1. Make the replacements 

G* . ^ G. . for i=l,2, . . . ,A-1 
A-i+1 A-i 

and set 

(^. — g ( s ) » 

s 

step Make the replacement 

f(s) T. ip(r,|b(s',s))f(s') for all s 

s'cP(s) 

and then return to step 1. 

[NOTE: For simplicity we have omitted the obvious "end game" modifications needed 

when t>L which of course are necessary only if a finite frame length is used. It 
should also be pointed out that our "trick" of storing the A past values of the 
g(s) summation actually results in a "true decoding delay" of 2A branches since 
we require the use of in step 2 at time t (in the algorithm) when a^_^ is 

decoded. To reduce the "true decoding delay" to A branches requires the storage 
of A+2 rather than 2 real numbers per state since f(s) must be updated by A 
branches and the A-1 previous values of f(s) stored for each state or, alternatively, 
the storage of 3 real numbers and considerable extra computation within the algorithm.] 
The algorithm as given above is directly suited for software implementation. 

It should be noted that steps 2, 3. and 5 call for ^ addition and multiplication 
of the computed probabilities so that floating-point arithmetic would normally 
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be chosen for the calculation. [This contrasts with the Viterbi algorithm as will 
be seen in the next section.] For each t, a total of A+1 calculations are made 
in steps 2 and 5 each involving a sum over all 2*^ states [whereas, as we shall 
see, only one similar calculation is needed for the Viterbi algorithm.] 

A hardware realization of the real-time minimum-bit-error algorithm could be 
made employing 2^ small computational, units (or CU’s) each of which corresponds 
to an encoder state s. Each such unit would receive f(s’) and g(s') from the 
two CU’s corresponding to the two states s’ which are predecessors of s and, with 
the aid of the received branches as inputs, would compute new values of f(s) and 
g(s) and pass these values on in turn to the two CU's corresponding to the states 
for which s is a predecessor. Each CU would execute A+1 computational cycles for 
each t [as compared to 1 cycle for the CU’s in a hardware Viterbi decoder.] 


IV. Real-Time Viterbi Decoding 

As mentioned in section I, the Viterbi decoding algorithm chooses 
be the information sequence which maximizes the conditional probability 


^^^[l L]l^[l L+ ]^' following decoding rule, which we call real-time ViterM 

decoding , is the natural modification of this rule to satisfy the ’’real-time 
constraint” that a^ be decoded from t+A]* digit in the 

information segment t+A] maximizes the conditional probability 

^^-[l,t+A]l^l,t+A]^‘ 


In keeping with our previous notation, we let 
^[t,f] "" ^^t’^t+l****’^t'^ 

denote the sequence of encoder states from time to to time t inclusive . It 
follows from (3) that and uniquely determine one another and, 

moreover, that a^ is the first digit of Hence we may rephrase the real-time 

Viterbi decoding rule as: Choose a^ as the first digit of the state in the 

state sequence S|-^ t+^+1] maximizes the conditional probability 
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P(s 


[l»t+A+l]'^l,t+A] 


) 


P(j 


[l,t+A+l]*-[l,t+A] 


P(rr, ^.,i) 


( 20 ) 


'Hl,t+A] 

Since the denominator on the righthand side of (20) is independent of ^Jp^t+A+l]’ 
we can equivalently maximize the numerator alone. 

To obtain a recxirsion for the numerator in (20) we use the same arguments as 
in Section II which, for t > 2, give 


^ ip(r, 1 

0 otherwise. ^2i) 


Equation (21 ) is our desired recursion. The initial condition to be used for 

t=2 is , 

4 P(rjb(0.s„)) if OeP(s„) 

( 22 ) 




2 '-1 
0 


otherwise. 

Just as for the ordinary Viterbi algorithm, the key to the efficient implemen- 
tation of real-time Viterbi decoding is the fact [readily seen from (21 )] that 
the best state sequence [in the sense of maximizing the joint probability on the 
lefthand side of (2l)] Sj-^ with must be the extension of the best 

sequence Sr_ with s =s ' or s =s ' ^ where s’ and s’’ are the predecessors of s, 
LljtJ t t 

Hence at each time t the only storage required for each state is the best sequence 
to that state. Actually, since real-time Viterbi decoding requires only knowledge 
of the first digit of the state A states previously along the sequence, we need 
store only A bits per state together with the joint probability needed for the 
recursion in (2l). We write B^(s), l£i£A, and h(s) for these stored quantities 
which at time t (in the algorithm) have the values 
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*’(®)"P(s*[i,t+l]*r[l,t]) 

where s*^ best path 

first digit of state s . 
/ \ XtX— X, 

BfCs) = 

in the path 


(23) 


{2h) 


Then we may state: 

The Real-Time Viterbi Decoding Algorithm ; 

Step 0 : Set t=l, set h(s)= -|P(r^ [h (^,s ) ) for the two states s having £ as a 

predecessor, and set h(s)=0 otherwise. Set B^(s)=0 for l£i^A and all states s. 
Step 1 : If t < A, go to Step 2. Otherwise set 

A 

where s is the state for which h(s) is maximxun. 

Step 2 : Increase t hy 1. Make the replacement i=l,2,..., 

A-1 and for all s . 

Step 3 : For each s, make the replacement h(s) ~ P(r^ lh(s ’ ,s ) )h(s ' ) 

where s’ is the predecessor of s which maximizes the replacing quantity and set 
Bi(s) = first digit of s’. 

Then retxirn to step 1. 

The algorithm Just given is directly suited for software implementation. 

Since the algorithm calls for only multiplication of the computed probabilities 
and selection of a maximum, logarithms may be used with the result that only 
computer additions are required and hence fixed-point arithmetic would normally 
be chosen for the calculation. For eact t, only one maximum over all states need 
be taken, an operation equivalent in complexity to a sum over all states as is 
required A+1 times in the algorithm of the preceeding section. In a hardware 
realization of the real-time Viterbi algorithm, the CU corresponding to state S 
would receive h(s’) from the two CU’s corresponding to the two predecessors s' 
of s and, with the aid of the received branch, would compute the new value of h(s) 
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pass this value on in turn to the two CU's for the states having s as a 
predecessor. Each CU would execute only one computational cycle for each time 
unit t and would he somewhat simpler than the CU's described in the preceeding 
section since only one quantity, h, (rather than two, f and g) would he processed 
and only additions need he performed. 

It should he emphasized that real-time Viterhi decoding may he used for 
L=«>, i.e. when the convolutional encoder is not periodically resynchronized. 
Moreover, ordinary Viterhi decoding can he considered the special case of real-time 
Viterhi decoding for finite L when A=L+m-l. 

V. Simulation Results 

To evaluate the performance of real-time miniminii-hit-error probability de- 
coding (hereafter called RTMBEP decoding), a rate R— 1/2 convolutional coding system 
was implemented for a simulated additive white Gaussian noise (AWGW) channel with 
8-level output quantization and with binary antipodal signalling. The results of 
this simulation are given in Table I where is the’ energy per information bit, 

Nq is the one-sided noise power spectral density, and K=m+1 is the encoder con- 
straint length measured in information hits. 

The simulation reported in Table I shows that the RTMBEP decoding algorithm 
gives a noticeable improvement in P^^, compared to real-time Viterhi decoding and 
ordinary Viterhi decoding only for small the improvement being about .2dB 

and .i+dB respectively when is 0 dB for K=3. We conclude that the slight 

improvement in P_„ for RTMBEP decoding would generally not justify the added 
BE 

complexity relative to real-time Viterhi decoding. 

In Table II, we show the effect of the decoding delay A on P^^ for K=3. 

From this table we see that a A of about 3K is sufficient for near optimal 


performance. 
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“o 

K 

L 

No. Frames 
Decoded 

A 

RTMBEP 

Decoding 

p 

BE 

Real-Time 

Viterbi 

Ordinary 

Viterbi 

OdB 

3 

2400 

1 


9 

.109 

.113 

.119 

2dB 

3 

2400 

2 


9 

.0165 

.0186 

,0173 

UdB 

3 

2400 

5 


9 

.00083 

.00083 

,00083 

OdB 

5 

2400 

1 


19 

.154 

,169 


2dB 

5 

2400 

2 


19 

.0165 

.0184 


UdB 

5 

2400 

5 


19 

.00033 

.0033 


Table I. 

Results of Decoding for a 
Gaussian Noise Channel 

Simulated Additive 

White 



“0 

K . 

L 

No . Frames 
Decoded 

A 

P 

BE 

OdB 

3 

1200 

1 

4 

.133 

OdB 

3 

1200 

1 

5 

.125 

OdB 

3 

1200 

1 

7 

.113 

OdB 

3 

1200 

1 

9 

.109 

OdB 

3 

1200 

1 

19 

.106 

OdB . 

3 

1200 

1 

29 

.102 

2dB 

3 

1200 

2 

4 

.0325 

2dB 

3 

1200 

2 

9 

.0169 

2dB 

3 

1200 

2 

19 

.oi4o 


Table II. Effect of Decoding Delay A for RTMBEP Decoding for a 
Simulated Additive White Gaussian Noise Channel 
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VI . Erasure 

Although RTMBEP decoding is not an attractive alternative to Viterbi decoding 
(either real-time or ordinary) for hard-decision decoding , it has good potential 
for use in soft-decision decoding such as in an inner coding system for con- 
catenated decoding (See [U] for an example of a concatenated scheme using an inner 
convolutional system.) This follows from the fact that the a posteriori probability 
P(at_^=0|r[i j)=If(s)/G^ (25) 

is directly computed (in step 3) by the algorithm so that the quality of the 
decoding decision for a^ ^ is directly known. 

To indicate the potential of RTMBEP decoding for soft-decision decoding, we 
consider the simplest case, for the AWGW channel viz. erasure decoding where 
the decision of a is either 0, 1 or E where E is an erasure symbol. The over- 
all convolutional coding system then converts the channel to a binary symmetric 
erasure channel (BSEC) as shown in Figixre 1 where q is the erasure probability 
and p is the 


1-P-q 



Fig, The Binary Symmetric Erasure Channel (BSEC ) Resulting from 
Erasures Decoding 


crossover probability . It whould be stressed that the BSEC model applies for a 
single decoding decision only since the encoder memory introduces memory into the 
overall channel. Nonetheless, the capacity C of the memoryless BSEC is a lower 
bound on the capacity of the true channel and hence a meaningful measure of the 
quality of the erasures decoding. 


lU 


The natural erasures rule for RTMBEP decoding is: Choose when 


a < P(a^ 1 ) < ~ + a . 


(26) 


‘t-A ''*-[1,1]^ ■ 2 

In Table III, we show the results of RTMBEP erasures decoding on the AWGN channel 
when a in (26) is chosen to maximize the capacity C of the raemoryless BSEC , 

[it is interesting to note that the optimum a in all cases corresponded to the 
case when 1/3 of the erased digits would have "been converted to errors in hard- 
decision decoding while 2/3 of the erased digits would have been correctly de- 
coded.] We also show the capacity for hard decisions, i.e. q.=0, as a standard to 
evaluate the gain for erasures decoding. The increase in capacity is seen to be 
.8 dB and 1.0 dB for K=3 and K=5 respectively. \Ie conclude that RTMBEP decoding 
has considerable potential for use in soft-decision applications for convolutional 
codes. 


“o 

K 

L A 

No. Frame 
Decoded 

■ P 

q 

Capacity C 
of BSEC 

Capacity for 
Hard Decisions 

OdB 

3 

1200 9 

1 

.0284 

.2k& 

.577 

.li80 

OdB 

5 

1200 19 

1 

.0342 

.327 

.478 

.380 

Table 

III: 

The Effectiveness of RTMBEP Erasures 
Additive White Gaussian Noise Channel 

Decoding for the 



It should be remarked that there seems to be no simple way to do effective 


erasures decoding with either real-time or ordinary Viterbl decoding since there 
is no simple way to determine the quality of the individual bit decoding decisions. 
For this reason, erasure of the entire frame has been used previously when Viterbi 
decoding is used for the inner decoder in a concatenated system [4]. This requires 
the interleaving of many frames to obtain satisfactory performance which is a 
substantial complication of the overall concatenated system which is avoided when 
individual bit decoding decisions can be effectively erased. 


15 


VII* Summary and Remarks 

We have given a fairly comprehensive treatment of optimum real-time decoding 
of convolutional codes and introduced an algorithm to minimize the decoding hit 
error probability. We also stated a real-time Viterbi decoding algorithm which, 
although not previously given in the literature, is probably the form of the 
Viterbi algroithm which has actually been used in many previous investigations. 

We remark that, although we considered only rate R=l/n convolutional codes, 
our discussion is easily generalized to the case R=k/n where k>l. In this case 

is the vector of information bits at time t. The RTMBEP decoding algorithm as 
we have given it becomes, mut at i s mut andi s , the algorithm which minimizes the 


symbol error probability 


Since a. contains only k bits , it follows that 
"D 

PsE ^ PbE i ^SE 

and since k is always small in applications there seems to be no practical 


justification for a decoding algorithm which minimizes when k > 1. 

Our major conclusion from simulations of the RTMBEP decoding algorithm is 
that, although it does not reduce P^^ sufficiently to be a practical alternative 
to Viterbi decoding in hard-decision applications, the fact that it provides a 
direct measure of the quality of its decoding decisions makes it an attractive 


candidate for the inner decoder in concatenated coding systems . 
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